1. Field of the Invention
The present invention relates to an image attribute altering device based on electronic watermark data, and particularly to an image attribute altering device based on electronic watermark data and an electronic watermark embedding device for embedding (inserting) electronic watermark data in which copyright owner ship is authenticated, infringement of copyright is easily searched, and electronic watermark (which is also referred to as digital watermark) which is used for audio, images video, multimedia, etc. is applied to provide specific attribute to electronically watermarked audio, images, etc., thereby restricting the copies of the audio, images, or so like.
2. Description of the Related Art
Recently, electronified media such as audio, images, video, multimedia, etc. have been increased, and also there has been required a guarantee system which can easily check intellectual proprietary rights such as copyright, etc. and data sources. An encryption system in which data are temporarily enciphered and only a proprietor can decipher the enciphered data is effective as a simple checking method as described above, and a method of subjecting original media to electronic watermarking has been known as an encryption system as described above
Further, from the viewpoint of protecting the copyright, there has been generalized that an attribute is provided to an original image to prohibit the original image from being copied or allow the original image to be copied only once or specific several times.
In order to alter the attribute of the image as described above, an attribute to be embedded during a vertical blanking period is generally altered. If the alteration of the attribute is allowed to be easily performed, the protection of the copyright cannot be sufficiently achieved. Accordingly, it has been required to prevent unjust copy of digital images on the basis of specific restricting conditions.
With respect to the attribute inserted during the vertical blanking period, it has been proposed in Japanese Laid-open Patent Application No. Hei-8-275127 that an identification signal to be inserted during the vertical blanking period (which is referred to as xe2x80x9cVBI signalxe2x80x9d, xe2x80x9cvideo IDxe2x80x9d or the like. In the following description, it is referred to as xe2x80x9cVBI signalxe2x80x9d) is used to prohibit or allow the copy of analog video signals of 525 line/60 field. The VBI signal is inserted into each of 20-th (first field) and 283rd (second field) horizontal periods (1H) during a vertical blanking period in video signals of one frame, and 20-bit reference and 20-bits digital signal are inserted into an effective video signal area after a horizontal synchronous signal and a color burst signal. CGMS (Copy Generation Management System) information representing copy allowance/prohibition is transmitted by 2 bits of the 20-bit digital signal.
The method of transmitting the CGMS information by using the VBI signal described above is applicable to analog signals such as 525-line standard video signals in NTSC system, 625-line standard video signals in PAL system or high-definition video signals such as high-vision signals or the like. However, the VBI signal is inserted to a predetermined place, and it can be easily removed by image processing such as filtering or the like, so that unjust copying of data cannot be prevented.
Therefore, there has been considered a method of embedding (inserting) electronic watermark data into an original image to prevent unjust use and unjust copy of the digital image.
Two kinds of watermark data, that is, visible electronic watermark data and invisible electronic watermark data are proposed as electronic data to digital images.
The visible electronic data is obtained by synthesizing special characters or symbols for images so that the characters or symbols are visible. It induces deterioration of image quality, however, has such an effect that it makes a visible appeal to users of digital images to prevent unjust flow of the images.
Japanese Laid-open Patent Application No. Hei-8-241403 discloses an embedding (inserting) manner of visible electronic watermark data. This publication discloses a method of forming a electronic watermarked image, which comprises a step of supplying a digital original image, a step of supplying a digital watermark image, a step of forming a watermarked image by superposing the watermark image on the original image without altering chromaticity of pixels of the original image when the watermarking is carried out, thereby placing the visible watermark on the digital image. The electronic watermarked image forming step contains a step of correcting pixels corresponding to the original image by altering the brightness (not color) for each of pixels which are not xe2x80x9ctransparentxe2x80x9d in the watermarked image. In this method, when the visible electronic watermark data are combined with the original image in this method, the electronic watermark data are combined with the original image so that only the brightness of the pixels corresponding to the opaque portion of the electronic watermark data are varied and no color component is varied. At this time, the scaling value of the variation of the brightness component of the pixels is determined in accordance with the color component, random number, the pixel value of the electronic watermark data. The conspicuousness of the watermark is determined by the scaling value.
Further, the invisible electronic watermark data are embedded into the original image so that much attention is paid to prevention of deterioration in image quality, and it is visually insensible because it has little deterioration in image quality.
As described above, if specific information with which the identification of a copyright holder is enabled is embedded as the electronic watermark data, the copyright holder can be identified by detecting the electronic watermark data even after an unjust copying is carried out. Further, if copy prohibiting information is embedded, when a reproducing device detects the copy-prohibiting information, a user may be made to be informed of the fact that the data are copy-prohibited data, or a copy preventing mechanism in the reproducing device may be drive to restrict the copy of the data to VTR or the like.
In order to embed the invisible electronic watermark data into digital images, there is known a method of embedding specific information as electronic watermark data into a portion having little effect on the image quality, such as LSB (Lease Significant Bit) of pixel data. However, even when the above method is used, it is still easy to remove the electronic watermark data from images. For example, the information of LSB of pixels would be lost if a low pass filter is used, and also the image compression processing aims to reduce the total data amount by omitting the information amount of such a portion having little effect on image quality. Therefore, the electronic watermark data are lost by the image processing, and thus there is such a problem that it is difficult to re-detect the electronic watermark data.
A method of subjecting an image to frequency conversion and embedding electronic watermark data into an area where the frequency component of a video signal after the frequency conversion is strong has been also proposed as another example of the method of subjecting an image to frequency transformation by using a low pass filter (xe2x80x9cNikkei Electronicsxe2x80x9d, 1996. 4. 22 (no. 66), p 13). According to this paper, as a method of installing ID information into a digital work, inherent ID information is installed every purchaser of a work (moving pictures, still pictures, photographs and sounds are targeted), and the ID information is used to expose works (publications, etc.) which were unjustly copied by persons other than the purchasers, and the electronic watermark data are embedded into the frequency components of the original works. Therefore, the electronic watermark data are not lost through the image processing such as compression processing, filtering or the like. Further, by using a normal distribution type random numbers as electronic watermark data, the interference between the electronic watermark data is prevented, and the destruction of the electronic watermark data is made hard without having no effect on the overall image.
FIG. 7 shows the construction described in this paper. The method of embedding the electronic watermark data according to the above method is as follows. First, an original image 701 is converted to frequency components by using DCT (Discrete Cosine Transformation) 703. Data of n which show high amplitude in the frequency area are selected, and they are represented as follows:
f(1), f(2), . . . , f(n)
w(1), w(2), . . . , w(n) of the electronic watermark data 701 are selected from a normal distribution whose average and dispersion are equal to 0 and 1 respectively, and the following equation is calculated for each (i) in electronic watermark data embedding means 704:
F(i)=f(i)+xcex1xc3x97|f(i)|xc3x97w(i)
According to the electronic watermark data embedding means 704, the output (f(i)) of DCT 703 is multiplied by the electronic watermark data 702 (w(i)) in a multiplier 305, and the multiplication result (|f(i)|xc3x97w(i)) is multiplied by a scaling element a 706 in a multiplier 707. The multiplication result (xcex1xc3x97|f(i)|xc3x97w(i)) is added with the output of DCT 7 in an adder 708 to obtain F(i).
Finally, by inverse DCT (inverse Discrete Cosine Transform) 709, an output image 710 in which the electronic watermark data are embedded is obtained from the frequency components obtained by converting f(i) to F(i).
Detection of the electronic watermark data is performed as follows. In this detection method, an original image 801 and electronic watermark data candidates w(i) (i=1, 2, . . . , n) 809 must be known.
Referring to FIG. 8, a target image 802 containing the electronic watermark data is converted to frequency components by using DCT 804 to obtain the following frequency components: F(1), F(2), . . . , F(n). Further, the original image 801 is also converted to frequency components by using DCT 803 to obtain the following frequency components: f(1), f(2), . . . , f(n). On the basis of f(i) and F(i) thus obtained, the electronic watermark data estimation value W(i) is calculated and extracted according to the following equation (f(i) is subtracted from F(i) in a subtracter 806, and then the subtraction result is divided by f(i)) in a divider 807) in the electronic watermark data detection means 805:
W(i)=(F(i)xe2x88x92f(i))/f(i)
Subsequently, statistical similarity C is calculated according to the following equation by using the vector inner product between the output W(i) of the electronic watermark data detection means 805 and the electronic watermark data w(i) in an inner product calculator 808:
C=Wxc3x97w/(WDxc3x97wD)
Here,
W=(W(1), W(2), . . . , W(n)),
w=(w(1), w(2), . . . , w(n))
WD=absolute value of vector W
wD=absolute value of vector w
Statistical similarity judgment means 810 judges that the electronic watermark data concerned are embedded if the statistical similarity C is equal to or more than a specific value.
If the electronic watermark data are embedded into an image by using the above method, this is effective to a case where a copyright proprietor owing an original image carries out the detection processing on digital image data which seems to be unjustly copied.
The above method needs the original image, and thus it is effective to only the case where the copyright proprietor owning the original image performs the detection processing on image data which seem to be unjustly copied, however, the reproducing device of each terminal cannot perform the detection processing of electronic watermark data because it has no original image. Therefore, a method obtained by improving the above method so that it is applicable to terminal processing, particularly an MPEG system has been proposed.
Referring to FIG. 9, according to this improved method, an original image 901 is divided into blocks 902 of 8 pixelsxc3x978 pixels, and the embedding and extracting processing of electronic watermark data is performed by using the block 902 as a processing unit.
Electronic watermark data embedding means 905 performs the frequency conversion through MPEG coding processing by using DCT 903. The AC components in a frequency area are represented by f(1), f(2), . . . , f(n) in increasing order of frequency component. Further, the electronic watermark data 904: w(1), w(2), . . . , w(n) are selected from a normal distribution having an average of 0 and a dispersion of 1, and the following equation is calculated for each i in the electronic watermark data embedding means 905:
F(i)=f(i)+xcex1xc3x97avg(f(i))xc3x97w(i)
Here, xcex1 represents a scaling element, and avg(f(i)) represents a partial average obtained by averaging the absolute points at three points in the neighborhood of f(i). Specifically, f(i) is subjected to partially averaging operation in a partial average unit 906, the partial average is multiplied by the electronic watermark data w(i) 904, the multiplication result avg(f(i))xc3x97w(i) is multiplied by a constant a in a multiplier 909, and the multiplication result xcex1xc3x97avg(f(i))xc3x97w(i) is added with the output of DCT 903 in an adder 910. f(i) is replaced by F(i), and the subsequent processing of the MPEG coding processing is carried out. The output F(i) of the electronic watermark data embedding means 905 is quantized according to a quantizing table 912 by a quantizer 911 and then coded in a encoder 913 to obtain MPEG data.
Subsequently, the detection of the electronic watermark data is carried out for the MPEG data by the following method. In the following detection method, no original image is needed, and it is sufficient that the electronic watermark data w(i) (i=1, 2, . . , n) has been known.
Referring to FIG. 10, the MPEG data 1001 are decoded in a decoder 1002, and then inversely quantized in an inverse quantizer 1003 to be expanded. Therefore, the inversely quantized data are subjected to inverse discrete cosine transform in an inverse DCT unit 1004 to obtain electronically watermarked image data 1005. The image data thus obtained are supplied to a display unit, whereby a display image containing an electronic watermark is obtained.
The electronic watermark data F(i) in the frequency area of the block after the inverse quantization of the MPEG expansion processing is completed is represented by F(1), F(2), . . . , F(n) in increasing order of frequency component. The average value of the absolute values at three points in the neighborhood of F(i) is calculated as a partial average avg(F(i)) by a partial average unit 1007, and then F(i) is divided by the partial average avg(F(i)) in a divider 1008 to obtain electronic data w(i) as follows:
W(i)=F(i)/avg(F(i))
Further, the sum WF(i) of the W(i) values of one image is calculated as follows in an adder 1009: WF(i)=W(1)+W(2)+ . . . +W(n), and the above sum is calculated for each (i). Subsequently, WF(i) and the electronic watermark data w(i) 1011 are input to an inner product calculator 1010 to calculate the inner product therebetween, and the statistical similarity C is calculated by using the vector inner product as follows:
C=WFxc3x97w/(WFDxc3x97wD)
Here,
WF=(WF(1), WF(2), . . . , WF(n))
w=(w(1), w(2), . . . , w(n)),
WFD=absolute value of vector WF
wD=absolute value of vector w
If the statistical similarity C is above a predetermined specific value, the statistic similarity judgment means 1012 judges that the electronic watermark data concerned are embedded.
FIG. 11 shows a detection method of electronic watermark data of an image after the MPEG expansion processing is completed. Image data 1101 are subjected to discrete cosine transform in a DCT unit 1102, and the image data thus transformed are represented by F(1), F(2), . . . , F(n) in increasing order of frequency component in the frequency area of the image block by the electronic watermark data detection means 1103. The average value of the absolute values at three points is calculated as a partial average avg(F(i)) in a partially averaging unit 1104, and then the electronic watermark data W(i) is calculated as follows:
W(i)=F(i)/avg(F(i))
Further, the sum WF(i) of W(i) of one image is calculated every (i) in an adder 1105. Subsequently, the statistical similarity C between the electronic watermark data w(i) 1106 and WF(i) is calculated according to the following equation by using the vector inner product in an inner product calculator 107:
C=WFxc3x97w/(WFDxc3x97wD)
Here,
WF=(WF(1), WF(2), . . . , WF(n)),
w=(w(1), w(2), . . . , w(n)),
WFD=absolute value of vector WF
wD=absolute value of vector w
If the statistical similarity C is above a specific value, a statistical similarity judgment means 1109 judges that the electronic watermark data concerned are embedded.
The electronic watermark data are detected from an image containing the electronic watermark data concerned and then the electronic watermark data concerned are subtracted from the image, whereby an original image can be obtained. In this case, any person other than the copyright proprietor can also obtain the original image by using this detection device, and thus the reproduction of the original image which is contrary to a copyright-proprietor""s will is possible. Therefore, a further improved encryption technique or enhanced secrecy has been required.
The present invention has been implemented in view of the foregoing situation, and has an object to further enhance secrecy for original media such as an original image or the like, clarify a copy-prohibited range and provide an attribute to allow only copies which are subject to a copyright-proprietor""s will.
In order to attaint the above object, an image attribute altering device according to a first aspect of the present invention includes image attribute detection means for detecting an input image (attribute xe2x80x9caxe2x80x9d) and the attribute of the input image, and image attribute altering means for altering the image attribute in the input image in accordance with the attribute detection result of the input image, whereby an output image (attribute xe2x80x9cBxe2x80x9d) having a new attribute is output.
Further, in order to attain the above object, an electronic watermark embedding device according to a second aspect of the present invention includes discrete cosine transforming means for subjecting an input image to discrete cosine transform, electronic watermark embedding means for inserting electronic watermark data into the data obtained by the discrete cosine transforming means, electronic watermark detection means for detecting whether the electronic watermark data to be inserted by the electronic watermark embedding means have been already inserted in the input image, and selection means for make the input image through if it is detected by the electronic watermark detection means that the electronic watermark data has been already inserted in the input image, or embedding new electronic watermark data into the input image if it is detected by the electronic watermark detection means that any electronic watermark data has not yet been embedded in the input image.